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Abstract 

It is known that a best low-rank approximation to multi-way arrays or higher-order tensors 
may not exist. This is due to the fact that the set of multi-way arrays with rank at most 
R is not closed. Nonexistence of the best low-rank approximation results in diverging rank-1 
components when an attempt is made to compute the approximation. Recently, a solution to 
this problem has been proposed for real / x J x 2 arrays. Instead of a best rank-i? approximation 
the best fitting Generalized Schur Decomposition (GSD) is computed. Under the restriction of 
nonsingular upper triangular matrices in the GSD, the set of GSD solutions equals the interior 
and boundary of the rank-i? set. Here, we show that this holds even without the restriction. We 
provide a complete classification of interior, boundary, and exterior points of the rank-i? set of 
real I x J x 2 arrays, and show that the set of GSD solutions equals the interior and boundary 
of this set. 
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1 Introduction 

This paper is an addendum to Stegeman and De Lathauwer [11] who study the following subject. 
Let o denote the outer-product, and define the outer-product rank of Y G j^^x-'x^ ^s 

R 

rankn(Y) = minjii | Y = ^^x^ o o z^} . (1-1) 

r=l 

Let 

Snil, J, K) = {Ye M^x-^x-^ I ranko(Y) < R} , (1.2) 

and let Sr{I, J, K) denote the closure of Sr{I, J, K), i.e. the union of the set itself and its boundary 
points in R^xJxX_ 

Let Z G j^-fxixii" g^^^ II . II (igjiQ^g ii^Q Frobenius norm on IR-^x^xis:^ Consider the following 
low-rank approximation problem. 

min{||Z-Y|| I Yg5^j(/,J,K)}. (1.3) 

Assuming ranko(Z) > R, an optimal solution of (|1.3p will be a boundary point of the set Sr{I, J, K). 
However, the set Sii{I, J,K) is not closed for R> 2, and problem p.3p may not have an optimal 
solution due to this fact; see De Silva and Lim [2]. Nonexistence of an optimal solution results in 
diverging rank-1 components when an attempt is made to compute a best rank-i? approximation, 
see Krijnen, Dijkstra and Stegeman |5J. In order to overcome this fallacy, [2j proposed to consider 
instead 

min{||Z-Y|| I YG5R(I,J,i^)}. (1.4) 

Note that if (jl.Sp has an optimal solution, then it is also an optimal solution of (jl.4p . To solve 
problem (jl.4p . we need to characterize the boundary points of Sr{I, J, K) and we need an algorithm 
to find an optimal boundary point. For R = 2, the boundary points are determined in and an 
algorithm to solve (jl.4p is proposed in Rocci and Giordani [7]. For a general approach to obtain 
an optimal solution to (jl.4p from an attempt to solve (jl.3p . see Stegeman |10j . 
In Stegeman and De Lathauwer [11] the case K = 2 is considered. Let 
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■) = {Y E M- 



/X Jx2 



Yfc = Q.Rfc Qf, A: = 1,2 



with Q^Q„ = Qb = Ir 



and Rfc (i? x i?) upper triangular} 



(1.5) 



denote the set of arrays with a full Generalized Schur Decomposition (GSD). Here, (/ x J) 
denotes the kih. frontal slice of Y. Note that a GSD exists only for R < min(/, J). In [llj it is 
shown that the problem 



is guaranteed to have an optimal solution. Moreover, it holds that Vr{I, J, 2) = J, 2) under 

the restriction that only arrays are considered that have a GSD with Ri and R2 nonsingular. Also, 
a Jacobi algorithm (based on De Lathauwer, De Moor and Vandewalle [Ij) is presented for solving 
()1.6p . Hence, under the above restriction, for K = 2 problem (jl.4p can be solved by solving problem 



In this note we show that the restriction used in |JJJ is not necessary. That is, we prove that 
Vr{I, J, 2) = Sb{I, J, 2) holds for R < min(/, J). 

We use the notation (S,T,U) • Y to denote the multilinear matrix multiplication of an array 

Y € M-f ><-^x^ with matrices S {h x /), T ( J2 x J), and U {K2 x K). The result of the multiphcation 
is an I2 X J2 X K2 array. We refer to (I7, Ij, U) • Y with U {K x K) nonsingular as a slicemix. 

For later use we mention that, for nonsingular S, T, U, and X = (S,T,U) • Y, we have 
ranko(X) = rankn(Y) and X is an interior (boundary, exterior) point of Sr{I, J,K) if and only if 

Y is an interior (boundary, exterior) point of Sr{I, J, K). 



Here, we consider the case where the arrays have two I x I slices and the number of components 
equals /. In [llj only arrays are considered that have two nonsingular slices. In Proposition 12.21 
below we present a complete classification of / x / x 2 arrays into interior, boundary, and exterior 
points of the set Sr{I , J,2). This classification is used to show that P/(/, /, 2) = 5/(1,1,2) in 



min{||Z-Y|| I YePij(/,J,2)} 



(1.6) 



0. 



2 The case I = J = R 
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Theorem 12 .Si In the proofs of Proposition 12.21 and Theorem 12.31 we use the following lemma which 
extends the theory on real matrix pencils and may be of interest in itself. 

Lemma 2.1 Let Y E M^^-^^^ with I x I slices Yi and Y2 such that det(/i Yi + A Y2) = for all 
fi,XeR. ThenYeVi{I,I,2). 

Proof. As shown in Moler and Stewart [6] (see also Golub and Van Loan [3, Section 7.7.2]) there 
exist orthonormal Q and Z such that G = Q Y2 Z is upper triangular and F = Q Yi Z is quasi- 
upper triangular. That is, F is block-upper triangular where its diagonal blocks are 2 x 2 or 1 x 1 
in size. The proof is complete if we show that there exist orthonormal Q and Z such that Q F Z 
and Q G Z are upper triangular. 

Let F have diagonal blocks Fi, . . . ,Fm, where Fj is 2 x 2 or 1 x 1. Denote the corresponding 
diagonal blocks of G by Gj, i = 1, . . . ,m. We have 



det(/iYi + AY2) = J|det(/iFi AGi) = 0, for all /i,A G 

j=i 

This can only hold if 

for some / , det(// F/ -|- A G;) = , for all /Li, A G M . 



(2.1) 



(2.2) 



It is shown in Moler and Stewart [SI Section 5] that if Fj is a 2 x 2 block and det(Fj -|- AGj) =0 
for some A G M, then 2x2 orthonormal Q and Z can be found such that Q Fj Z and Q Gj Z are 
upper triangular. Hence, we may assume without loss of generality that if Fj is a 2 x 2 block, then 
det(Fi + XGi)^0 for ah A G M. 

Let index I be as in (|2.2p . From the discussion above it follows that we may assume that F/ is 
1x1. Hence, F and G have a zero on their diagonals in the same position. Suppose the common 
zero appears right after a 2 x 2 block Fj, i.e. 



Fi f 



0^ 
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* 


* 
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* 


* 












G. g 
0^ 
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* 












(2.3) 



Let zi G be orthogonal to the second rows of the matrices in (j2.3p . Then postmultiplying (|2.3p 
by any orthonormal Z = [zi|z2|z3] brings both matrices into upper triangular form and leaves the 



common zero in the same position. Analogously, if the common zero appears right before a 2 x 2 
block Fj, then premultiplying by a suitable orthonormal Q does the trick. 

If the common zero on the diagonals of F and G is not adjacent to a 2 x 2 block Fj, then 
we resort to simultaneously reordering the diagonal blocks of F and G (except the common zero) 
such that it is. It suffices to show that swapping adjacent 2x2 and 1x1 blocks is possible by 
orthonormal transformations. Let Fj be 2 x 2 and consider the 3x3 matrices 

F. f 

Swapping the diagonal blocks i and i + 1 by orthonormal transformations is possible if x, y G 
exist such that they satisfy the so-called generalized Sylvester equation (see e.g. Kressner [H Section 
5]): 

Fj X - /j+i y = f , Gj X - fifj+i y = g . (2.5) 

We may assume that /j+i and gi^i are not both zero. Let ^j+i / (the proof for /j+i 7^ is 
analogous). Then ()2.5p is satisfied for 

y = (G,x-g)/5m, x = (Fj-(/,+i/<7m)Gj)-^(f-(/,+i/5^+i)g). (2.6) 

Note that det(Fj — (/j+i/(7j_(_i) Gj) 7^ by assumption, and the solution (j2.6|) is unique. Hence, 
the diagonal blocks can be swapped. Analogously, it can be shown that a 1 x 1 block i and a 2 x 2 
block i + 1 can be swapped. This completes the proof. □ 

Proposition 2.2 Let Y G M^^^><2 

(a) // there exists a U nonsingular such that X = (I/, I/) U) • Y has nonsingular slice Xi, then 

(al) Y is an interior point of Si{1 , 1 ,2) if ^2'^i^ has I distinct real eigenvalues. 

(a2) Y is a boundary point of Sj{I,I,2) if ^^2^1^ has I real eigenvalues but not all distinct. 

(a3) Y is an exterior point ofSj{I, I, 2) z/X2Xj^"'^ has at least one pair of complex eigenvalues. 

(b) // there does not exist a U nonsingular such that X = (I/,I/,U) • Y has nonsingular slice 
Xi, thenY_ is a boundary point of Si {1, 1, 2). 
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Gj g 

0^ gi+i 



(2.4) 



Proof. The proofs of (a) follow from the fact that multilinear matrix multiplication leaves the prop- 
erty interior (boundary, exterior) point invariant, and application of Stegeman and De Lathauwer 
[m Lemma 3.1] which is due to Stegeman [8]. 

Next we prove (b). From Lemma l2.ll it follows that Y = {Qa,QbT^2) ■ where R has two 
upper diagonal slices. Below, we show that R is a boundary point of 5/(1, /, 2). Since Qa and Q;, 
are nonsingular, it follows that also Y is a boundary point of /, 2). 

It holds that det(/iRi + AR2) = for all /i, A € M, which implies that Ri and R2 have a zero 
on their diagonals in the same position. A small perturbation of the diagonals of Ri and R2 yields 
slices Hi (nonsingular) and H2, with H2Hj~^ (upper triangular) having / real eigenvalues, and 
H|| < e for any e > 0. Next, we show that it is possible to choose the perturbation such that 
HzH^;^ has a pair of identical eigenvalues. For simplicity, we assume that the diagonals of Ri and 
R2 contain one common zero. A proof for the general case is analogous. 

Let (Ri)ii = (R2)m = and set (Hi)^^ = 61 and (H2)m = '^2- This yields a nonzero eigenvalue 
62/61 for H2HJ~^ (assuming small perturbations of the other zeros on the diagonal of Ri, such that 
Hi is nonsingular). Unless stated otherwise, we only perturb the zero diagonal elements of Ri 
and R2. If, for some j 7^ i, (Ri)jj 7^ and (R2)jj 7^ 0, then let A = (R2)jj/(Ri)jj, and choose 
62 = X6i. This yields H2H^^ with two identical real eigenvalues 62/61 = A for any ^i > 0. If no 
common nonzero diagonal elements of Ri and R2 exist, then we proceed as follows. If, for some 
j ^ i, (Ri)jj 7^ and (R2)jj = 0, then let (H.2) jj = choose 61 = \/52 (Ri)jj and r] = \/^. 

This yields H2HJ~^ with two identical real eigenvalues 62/61 = r]/{'Ri)jj for any 62 > 0. If, for some 
j 7^ i, (Ri)jj = and (R2)jj 7^ 0, then let (H.i) jj = and choose 62 = V^(R2)jj and 7] = -/6i. 
This yields H2Hj^"'^ with two identical real eigenvalues 62/61 = (R2)jj/'n for any 61 > 0. Hence, it 
is possible to get H2Hj~^ with a pair of identical eigenvalues. 

By Proposition 12.21 (a2). the array H is a boundary point of Sj{I, I, 2). Since | |R — H| | < e for 
any e > 0, it follows that R can be approximated arbitrarily closely from Sj{I,I,2). Hence, we 
obtain R G Sj{I, /, 2). Moreover, since for any e > the array H is a boundary point of Sj{I, /, 2), 
it follows that R itself must be a boundary point of 5/(/, /, 2). □ 

We are now ready to present our result for / = J = R. 
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Theorem 2.3 It holds that Vj{I,I,2) = Si{I,I,2). 

Proof. First, observe that X = (I/, I/, U) • Y with nonsingular U has a full GSD if and only if Y 
has a full GSD. Indeed, a slicemix of upper triangular slices results in upper triangular slices. 

This observation, together with Stegeman and De Lathauwer |1H Lemma 5.1], yields the fol- 
lowing results for the arrays Y in Proposition 12.21 If Y satisfies (al) or (a2), then Y G Vi{I, I, 2). 
If Y satisfies (a3), then Y ^ Vi{I, I, 2). 

Lemma l2. II shows that an array Y satisfying (b) lies in Vi{I,I,2). Since (a)-(b) defines a par- 
tition of IR^^^^2, we have shown that Y G Vi{I, I, 2) if and only if Y G /, 2). This completes 
the proof. □ 



3 Extension to general /, J, R 

A GSD exists only for R < min(/, J). However, nonexistence of an optimal solution to problem (|1.3p 
for / X J X 2 arrays does not seem to occur for R > I or R > J; see Stegeman [9j. In Theorem 13.21 
below we show that Vr{I, J, 2) = Sr{I, J, 2) for R < min(/, J). This extends Theorem 12.31 In the 
proof of Theorem 13.21 we make use of Theorem 12.31 and the following lemma, which concerns an 
orthogonal equivalence between interior and boundary points of Sr{I, J, 2) and those of Sr^R, R, 2). 

Lemma 3.1 Let Y G R^^J^'^ with R < min(/, J). Then Y G Sr{I, J, 2) if and only if there exist 
S {IxR) andT {J x R) with S^S = T^T = I/j such thatY = (S, T, I2) -X wii/i X G ii, 2). 

Moreover, Y G Sr{I, J, 2) if and only ifXG Sr{R, R, 2). 

Proof. See [2, Theorem 5.2]. □ 
Theorem 3.2 Let R < min(I, J). It holds that Vr{I, J, 2) = Sr{I, J, 2). 

Proof. Let Y G Sr{I, J, 2). By Lemma O and Theorem ES] we have Y = (S, T, I2) • X with 
X G Sr{R, R, 2) = Vr{R, R, 2). This implies 

Yfe = SXfcT^ = (SQ,)Rfc(TQfef , k = l,2. (3.1) 
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Since the matrices SQa and TQ;, are column-wise orthonormal and is Rx R upper triangular, 
(IHH) implies that Y e Vr{I, J, 2). 

Next, let Y G Vr{I, J,2). Then Y^ = QaR-feQ^" ^ov k = 1,2, which is equivalent to 
Y = (Qa,Q6,l2)-R, where R e has two upper triangular slices. Bence^K G Vr{R,R, 2) = 

Sr{R, R, 2) by Theorem 12.31 An application of Lemma [3TT] yields Y G Sji{I, J, 2). This completes 
the proof. □ 



4 Conclusion 

We have shown that the set of / x J x 2 arrays with a full GSD of size R equals the closure of the 
set of / X J X 2 arrays with at most rank R. Also, we have provided a complete classification of 
interior, boundary, and exterior points of the latter set. This extends the theoretical results in 
which were limited to the case of nonsingular upper triangular matrices in the GSD. 
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